The lecture schedule will be updated as the term progresses.
| Date | Topic | Readings |
|---|---|---|
| Wed, Jan 15, 2020 | Course Intro : Text Classification and Sentiment Analysis [slides] [video] |
Jurafsky and Martin, Chapter 4 "Naive Bayes Classification and Sentiment"
Bo Pang, Lillian Lee and Shivakumar Vaithyanathan, Thumbs up? Sentiment Classification using Machine Learning Techniques Bo Pang and Lillian Lee, Movie Review Data |
| Mon, Jan 20, 2020 | Martin Luther King, Jr. Day Observed (no classes) | |
| Wed, Jan 22, 2020 | Text Classification Wrapup / Text Processing [slides] [video] |
Jurafsky and Martin, Chapter 2 "Regular Expressions, Text Normalization, and Edit Distance"
Andrew Yates, Arman Cohan, Nazli Goharian, Depression and Self-Harm Risk Assessment in Online Forums , Publicly available Reddit comments Optional: Marti A. Hearst, Automatic Acquisition of Hyponyms from Large Text Corpora Optional: Stephen Roller, Douwe Kiela, and Maximilian Nickel, Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora |
| Mon, Jan 27, 2020 | Text Processing Wrapup / Logistic Regression [slides] [video] |
Jurafsky and Martin, Chapter 5 "Logistic Regression"
Optional: Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 2). This book is free to download from Penn's campus. |
| Wed, Jan 29, 2020 | Logistic Regression Wrapup [slides] [video] |
Jurafsky and Martin, Chapter 5 "Logistic Regression"
|
| Wed, Jan 29, 2020 | N-gram Language Models [slides] [video] |
Jurafsky and Martin, Chapter 3 "N-gram Language Models"
Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, Jeffrey Dean, Large Language Models in Machine Translation |
| Mon, Feb 3, 2020 | Vector Semantics [slides] [video] |
Jurafsky and Martin, Chapter 6 "Vector Semantics"
Peter Turney and Patrick Pantel, From Frequency to Meaning: Vector space models of semantics Optional: Chris Potts, Video - Overview of distributed word representations Optional: Chris Potts, Video - Vector comparison for distributed word representations Optional: Chris Potts, Video - Matrix reweighting for distributed word representations |
| Wed, Feb 5, 2020 | Vector Semantics - part 2 [slides] [video] |
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean, Distributed Representations of Words and Phrases and their Compositionality
Yoav Goldberg and Omer Levy, word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method Omer Levy and Yoav Goldberg, Linguistic regularities in sparse and explicit word representations Optional: Julia Bazińska, Explore word analogies (online interactive demo) |
| Mon, Feb 10, 2020 | Vector Semantics - part 3 [slides] [video] |
Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama1, Adam Kalai, Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Optional: Aylin Caliskan, Joanna Bryson, Arvind Narayanan, Semantics derived automatically from language corpora contain human-like biases Optional: Omer Levy, How does Mikolov's word analogy for word embedding work? How can I code such a function? Optional: Tomas Mikolov, Scott Wen-tau Yih, and Geoffrey Zweig, Linguistic Regularities in Continuous Space Word Representations Optional: William L. Hamilton, Jure Leskovec, Dan Jurafsky, Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change Optional: Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou, Word embeddings quantify 100 years of gender and ethnic stereotypes |
| Wed, Feb 12, 2020 | No class. Chris is traveling for a DARPA PI meeting. | |
| Wed, Feb 19, 2020 | Neural Net Language Models [slides] [video] |
Jurafsky and Martin, Chapter 5 "Logistic Regression" (sections 5.2-5.6)
Jurafsky and Martin, Chapter 7 "Neural Networks and Neural Language Models" Optional: Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 4). This book is free to download from Penn's campus. Bengio, Ducharme, Vincent, A Neural Probabilistic Language Model Optional: Bengio, Ducharme, Vincent and Jauvin, A Neural Probabilistic Language Model (longer JMLR version) |
| Wed, Feb 19, 2020 | Neural Net Language Models - part 2 [slides] [video] |
Jurafsky and Martin, Chapter 9 "Neural Networks and Neural Language Models"
Optional: Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 14). Bengio, Ducharme, Vincent, A Neural Probabilistic Language Model Optional: Bengio, Ducharme, Vincent and Jauvin, A Neural Probabilistic Language Model (longer JMLR version) |
| Fri, Feb 21, 2020 | Drop Deadline | |
| Mon, Feb 24, 2020 | Part-of-Speech Tagging [slides] [video] |
Jurafsky and Martin, Chapter 8 "Part-of-Speech Tagging"
|
| Wed, Feb 26, 2020 | Constituency Grammars [slides] [video] |
Jurafsky and Martin, Chapter 12 "Constituency Grammars"
|
| Mon, Mar 2, 2020 | Grammars and Constituency Parsing [slides] [video] [video-part-2] |
Jurafsky and Martin, Chapter 12 "Constituency Grammars"
Jurafsky and Martin, Chapter 13 "Constituency Parsing" Lisbon Machine Learning School, CKY Demo Dragomir Radev, (Video) Classic Parsing Methods for Natural Language Processing Dragomir Radev, (Video) The Penn Treebank |
| Wed, Mar 4, 2020 | Statistical Constituency Parsing [slides] [video] |
Jurafsky and Martin, Chapter 14 "Statistical Constituency Parsing"
|
| Mon, Mar 9, 2020 | No Lecture - Spring Break | |
| Wed, Mar 11, 2020 | No Lecture - Spring Break | |
| Mon, Mar 16, 2020 | No Lecture - Spring Break Extended | |
| Wed, Mar 18, 2020 | No Lecture - Spring Break Extended | |
| Mon, Mar 23, 2020 | Updates to course policies / Wrap-up of Constituency Parsing / Dependency Parsing [slides] [video] |
Jurafsky and Martin, Chapter 14 "Statistical Constituency Parsing"
Jurafsky and Martin, Chapter 15 "Dependency Parsing" Dragomir Radev, Dependency Parsing Dragomir Radev, Statistical Parsing |
| Wed, Mar 25, 2020 | Logical Representations of Sentence Meaning [slides] [video] |
Jurafsky and Martin, Chapter 16 "Logical Representations of Sentence Meaning"
|
| Wed, Mar 25, 2020 | HW7 "Named Entity Recognition" due | |
| Mon, Mar 30, 2020 | Wrapup of Logical Representations of Sentence Meaning and Information Extraction [slides] [video] |
Jurafsky and Martin, Chapter 18 "Information Extraction"
|
| Mon, Mar 30, 2020 | Quiz due (covers Chapters 12-14) |
Jurafsky and Martin, Chapter 12 "Constituency Grammars"
Jurafsky and Martin, Chapter 13 "Constituency Parsing" Jurafsky and Martin, Chapter 14 "Statistical Constituency Parsing" |
| Wed, Mar 25, 2020 | Deadline to decide on term project versus weekly homework option. Please specify your preference by filling out this form. | |
| Fri, Mar 27, 2020 | Deadline to complete the IRB training for HW9, if you're doing the HW option. Please follow the instruction for obtaining data on the HW9 page. | |
| Wed, Apr 1, 2020 | Information Extraction and Semantic Role Labeling [slides] [video] |
Jurafsky and Martin, Chapter 20 "Semantic Role Labeling"
|
| Wed, Apr 1, 2020 | HW8 "Learning Hypernyms" due | |
| Fri, Apr 3, 2020 | Withdraw Deadline | |
| Mon, Apr 6, 2020 | Semantic Role Labeling and Encoder-Decoder Models [slides] [video] |
Jurafsky and Martin, Chapter 20 "Semantic Role Labeling"
Jurafsky and Martin, Chapter 10 "Encoder-Decoder Models, Attention, and Contextual Embeddings" |
| Mon, Apr 6, 2020 | Quiz due (covers Chapters 15 and 16) |
Jurafsky and Martin, Chapter 15 "Dependency Parsing"
Jurafsky and Martin, Chapter 16 "Logical Representations of Sentence Meaning" |
| Wed, Apr 8, 2020 | Encoder-Decoder Models and Neural Machine Translation [slides] [video] |
Graham Neubig, Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
|
| Wed, Apr 8, 2020 | Neural Machine Translation (recording of guest lecture from 2019) [slides] [video] by Graham Neubig |
Graham Neubig, Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
|
| Wed, Apr 8, 2020 | Project Milestone 1 or HW9 "Classify Depression" due | |
| Mon, Apr 13, 2020 | MT Evaluation and Cross-lingual Models of Word Embeddings [slides] [video] |
Shyam Upadhyay, Manaal Faruqui, Chris Dyer, Dan Roth, Cross-lingual Models of Word Embeddings: An Empirical Comparison
|
| Mon, Apr 13, 2020 | Quiz due (covers Chapters 18 and 20) |
Jurafsky and Martin, Chapter 18 "Information Extraction"
Jurafsky and Martin, Chapter 20 "Semantic Role Labeling" |
| Wed, Apr 15, 2020 | Dialogue Systems and Chatbots [slides] [video] |
Jurafsky and Martin, Chapter 26 "Dialogue Systems and Chatbots"
|
| Wed, Apr 8, 2020 | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [slides] [video] by Jacob Devlin |
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT&colon Pre-training of Deep Bidirectional Transformers for Language Understanding
|
| Wed, Apr 15, 2020 | Project Milestone 2 or HW10 "Neural Machine Translation" due | |
| Mon, Apr 20, 2020 | Text Generation with Neural Networks [slides] [video] by Daphne Ippolito |
Optional:
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin, Attention is All You Need
Optional: Jay Alammar, The Illustrated Transformer Optional: Jay Alammar, The Illustrated GPT-2 (Visualizing Transformer Language Models) Optional: Jay Alammar, The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) Optional: Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng, The Woman Worked as a Babysitter: On Biases in Language Generation |
| Mon, Apr 20, 2020 | Quiz due (covers Chapters 10 and 26) |
Jurafsky and Martin, Chapter 10 "Encoder-Decoder Models, Attention, and Contextual Embeddings"
Jurafsky and Martin, Chapter 26 "Dialog Systems and Chatbots" |
| Wed, Apr 22, 2020 | The Transformer Model [slides] [video] by Daphne Ippolito |
Optional:
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin, Attention is All You Need
Optional: Jay Alammar, The Illustrated Transformer Optional: Jay Alammar, The Illustrated GPT-2 (Visualizing Transformer Language Models) Optional: Jay Alammar, The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) |
| Wed, Apr 22, 2020 | Project Milestone 3 or HW11 "Perspective Detection" due | |
| Mon, Apr 27, 2020 | Question Answering [slides] [video] |
Jurafsky and Martin, Chapter 25 "Question Answering"
|
| Mon, Apr 27, 2020 | Quiz due (covers Chapters 22 and 25) |
Jurafsky and Martin, Chapter 22 "Coreference Resolution"
Jurafsky and Martin, Chapter 25 "Question Answering" |
| Wed, Apr 29, 2020 | Course Wrapup [slides] [video] | |
| Wed, Apr 29, 2020 | Project Milestone 4 or HW12 "BERT" due | |
| Wed, Apr 29, 2020 | Deadline to switch to Pass/Fail |