Skip to main content

The lecture schedule will be updated as the term progresses.

Date Topic Required Readings Supplemental Videos
Wed, Jan 16, 2019 Course Intro / Text Classification and Sentiment Analysis Jurafsky and Martin, Chapter 4 "Naive Bayes Classification and Sentiment"
Bo Pang, Lillian Lee and Shivakumar Vaithyanathan, Thumbs up? Sentiment Classification using Machine Learning Techniques
Bo Pang and Lillian Lee, Movie Review Data
Mon, Jan 21, 2019 Martin Luther King, Jr. Day Observed (no classes)
Wed, Jan 23, 2019 Text Classification Wrapup / Text Processing Jurafsky and Martin, Chapter 2 "Regular Expressions, Text Normalization, and Edit Distance"
Andrew Yates, Arman Cohan, Nazli Goharian, Depression and Self-Harm Risk Assessment in Online Forums
, Publicly available Reddit comments
Mon, Jan 28, 2019 Text Processing Wrapup Marti A. Hearst, Automatic Acquisition of Hyponyms from Large Text Corpora
Stephen Roller, Douwe Kiela, and Maximilian Nickel, Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora
Wed, Jan 30, 2019 N-gram Language Models [lecture notes] Jurafsky and Martin, Chapter 3 "N-gram Language Models"
Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, Jeffrey Dean, Large Language Models in Machine Translation
Mon, Feb 4, 2019 N-gram Language Models Wrapup Jurafsky and Martin, Chapter 3 "N-gram Language Models"
Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, Jeffrey Dean, Large Language Models in Machine Translation
Mon, Feb 4, 2019 Vector Semantics [lecture notes] Jurafsky and Martin, Chapter 6 "Vector Semantics"
Peter Turney and Patrick Pantel, From Frequency to Meaning: Vector space models of semantics
Chris Potts, Overview of distributed word representations (10 minutes)
Chris Potts, Vector comparison for distributed word representations (10 minutes)
Chris Potts, Matrix reweighting for distributed word representations (16 minutes)
Mon, Feb 11, 2019 Vector Semantics - part 2 Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean, Distributed Representations of Words and Phrases and their Compositionality
Yoav Goldberg and Omer Levy, word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method
Wed, Feb 13, 2019 Vector Semantics - part 3 Omer Levy and Yoav Goldberg, Linguistic regularities in sparse and explicit word representations
Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama1, Adam Kalai, Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Optional: Aylin Caliskan, Joanna Bryson, Arvind Narayanan, Semantics derived automatically from language corpora contain human-like biases
Optional: Omer Levy, How does Mikolov's word analogy for word embedding work? How can I code such a function?
Optional: Tomas Mikolov, Scott Wen-tau Yih, and Geoffrey Zweig, Linguistic Regularities in Continuous Space Word Representations
Optional: William L. Hamilton, Jure Leskovec, Dan Jurafsky, Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
Optional: Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou, Word embeddings quantify 100 years of gender and ethnic stereotypes
Wed, Feb 20, 2019 Logistic Regression Jurafsky and Martin, Chapter 5 "Logistic Regression"
Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 2). This book is free to download from Penn's campus.
Wed, Feb 20, 2019 No class today due to snow. Pleaes read ahead on neural network LMs. Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 4). This book is free to download from Penn's campus.
Bengio, Ducharme, Vincent, A Neural Probabilistic Language Model
Optional: Bengio, Ducharme, Vincent and Jauvin, A Neural Probabilistic Language Model (longer JMLR version)
Fri, Feb 22, 2019 Drop Deadline
Mon, Feb 25, 2019 Neural Net Language Models [lecture notes] Yoav Goldberg, Neural Network Methods for Natural Language (Chapters 4 and 9). This book is free to download from Penn's campus.
Bengio, Ducharme, Vincent, A Neural Probabilistic Language Model
Optional: Bengio, Ducharme, Vincent and Jauvin, A Neural Probabilistic Language Model (longer JMLR version)
Wed, Feb 27, 2019 Part-of-Speech Tagging and Sequence Models Jurafsky and Martin, Chapter 8 "Part-of-Speech Tagging"
Mon, Mar 4, 2019 No Lecture - Spring Break
Wed, Mar 6, 2019 No Lecture - Spring Break
Mon, Mar 11, 2019 POS tagging and Sequence Models Jurafsky and Martin, Chapter 8 "Part-of-Speech Tagging"
Wed, Mar 13, 2019 Guest lecture on BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Jacob Devlin Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Mon, Mar 18, 2019 POS Sequence Processing and Recurrent Networks Yoav Goldberg, Recurrent Neural Networks&colon Modeling Sequences and Stacks (Chapter 14). This book is free to download from Penn's campus.
Wed, Mar 20, 2019 Guest lecture on Cross-lingual Word Representations by Shyam Upadhyay Shyam Upadhyay, Manaal Faruqui, Chris Dyer, Dan Roth, Cross-lingual Models of Word Embeddings: An Empirical Comparison
Mon, Mar 25, 2019 Machine Translation and IBM word alignment models
Wed, Mar 27, 2019 Guest lecture by Graham Neubig on NMT Graham Neubig, Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
Mon, Apr 1, 2019 Question Answering Jurafsky and Martin, Chapter 23 "Question Answering"
Wed, Apr 3, 2019 Guest lecture on Automatic Speech Recognition by Eric Fosler-Lussier Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Brian Kingsbury, Tara Sainath, Deep Neural Networks for Acoustic Modeling in Speech Recognition
Yajie Miao, Mohammad Gowayyed, Florian Metze, End-to-end speech recognition using deep RNN models and WFST-based decoding
Fri, Apr 5, 2019 Withdraw Deadline
Mon, Apr 8, 2019 Guest lecture on Dialog Systems / Chatbots by João Sedoc Optional: Michael McTear, Zoraida Callejas, and David Griol, The Conversational Interface
Optional: João Sedoc, Dialog Systems Class
Wei Wu and Rui Yan, (video) Deep Chit-Chat tutorial: Deep Learning for ChatBots (2 hours and 45 minutes)
Wei Wu and Rui Yan, (slides) Deep Chit-Chat tutorial: Deep Learning for ChatBots
Wed, Apr 10, 2019 Information Extraction Jurafsky and Martin, Chapter 17 "Information Extraction"
Ellie Pavlick, Heng Ji, Xiaoman Pan, and Chris Callison-Burch, The Gun Violence Database: A new task and data set for NLP
Optional: Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, Jamie Taylor, Freebase: A Collaboratively Created Graph Database For Structuring Human Knowledge
Optional: Soren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives, DBpedia: A Nucleus for a Web of Open Data
Optional: Mausam, Michael Schmitz, Robert Bart, Stephen Soderland, and Oren Etzioni, Open Language Learning for Information Extraction
Mon, Apr 15, 2019 Formal Grammars of English Jurafsky and Martin, Chapter 10 "Formal Grammars of English"
Wed, Apr 17, 2019 Syntactic and Statistical Parsing Jurafsky and Martin, Chapter 11 "Syntactic Parsing"
Jurafsky and Martin, Chapter 12 "Statistical Parsing"
Mon, Apr 22, 2019
Wed, Apr 24, 2019 Final project presentations
Mon, Apr 29, 2019 Final project presentations
Wed, May 1, 2019 Final project presentations