Skip to main content

The lecture schedule will be updated as the term progresses.

Date Topic Readings
Wed, Jan 15, 2020 Course Intro / Text Classification and Sentiment Analysis [slides] [video] Jurafsky and Martin, Chapter 4 "Naive Bayes Classification and Sentiment"
Bo Pang, Lillian Lee and Shivakumar Vaithyanathan, Thumbs up? Sentiment Classification using Machine Learning Techniques
Bo Pang and Lillian Lee, Movie Review Data
Mon, Jan 20, 2020 Martin Luther King, Jr. Day Observed (no classes)
Wed, Jan 22, 2020 Text Classification Wrapup / Text Processing [slides] [video] Jurafsky and Martin, Chapter 2 "Regular Expressions, Text Normalization, and Edit Distance"
Andrew Yates, Arman Cohan, Nazli Goharian, Depression and Self-Harm Risk Assessment in Online Forums
, Publicly available Reddit comments
Optional: Marti A. Hearst, Automatic Acquisition of Hyponyms from Large Text Corpora
Optional: Stephen Roller, Douwe Kiela, and Maximilian Nickel, Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora
Mon, Jan 27, 2020 Text Processing Wrapup / Logistic Regression [slides] [video] Jurafsky and Martin, Chapter 5 "Logistic Regression"
Optional: Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 2). This book is free to download from Penn's campus.
Wed, Jan 29, 2020 Logistic Regression Wrapup [slides] [video] Jurafsky and Martin, Chapter 5 "Logistic Regression"
Wed, Jan 29, 2020 N-gram Language Models [slides] [video] Jurafsky and Martin, Chapter 3 "N-gram Language Models"
Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, Jeffrey Dean, Large Language Models in Machine Translation
Mon, Feb 3, 2020 Vector Semantics [slides] [video] Jurafsky and Martin, Chapter 6 "Vector Semantics"
Peter Turney and Patrick Pantel, From Frequency to Meaning: Vector space models of semantics
Optional: Chris Potts, Video - Overview of distributed word representations
Optional: Chris Potts, Video - Vector comparison for distributed word representations
Optional: Chris Potts, Video - Matrix reweighting for distributed word representations
Wed, Feb 5, 2020 Vector Semantics - part 2 [slides] [video] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean, Distributed Representations of Words and Phrases and their Compositionality
Yoav Goldberg and Omer Levy, word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method
Omer Levy and Yoav Goldberg, Linguistic regularities in sparse and explicit word representations
Optional: Julia Bazińska, Explore word analogies (online interactive demo)
Mon, Feb 10, 2020 Vector Semantics - part 3 [slides] [video] Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama1, Adam Kalai, Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Optional: Aylin Caliskan, Joanna Bryson, Arvind Narayanan, Semantics derived automatically from language corpora contain human-like biases
Optional: Omer Levy, How does Mikolov's word analogy for word embedding work? How can I code such a function?
Optional: Tomas Mikolov, Scott Wen-tau Yih, and Geoffrey Zweig, Linguistic Regularities in Continuous Space Word Representations
Optional: William L. Hamilton, Jure Leskovec, Dan Jurafsky, Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
Optional: Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou, Word embeddings quantify 100 years of gender and ethnic stereotypes
Wed, Feb 12, 2020 No class. Chris is traveling for a DARPA PI meeting.
Wed, Feb 19, 2020 Neural Net Language Models [slides] [video] Jurafsky and Martin, Chapter 5 "Logistic Regression" (sections 5.2-5.6)
Jurafsky and Martin, Chapter 7 "Neural Networks and Neural Language Models"
Optional: Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 4). This book is free to download from Penn's campus.
Bengio, Ducharme, Vincent, A Neural Probabilistic Language Model
Optional: Bengio, Ducharme, Vincent and Jauvin, A Neural Probabilistic Language Model (longer JMLR version)
Wed, Feb 19, 2020 Neural Net Language Models - part 2 [slides] [video] Jurafsky and Martin, Chapter 9 "Neural Networks and Neural Language Models"
Optional: Yoav Goldberg, Neural Network Methods for Natural Language (Chapter 14).
Bengio, Ducharme, Vincent, A Neural Probabilistic Language Model
Optional: Bengio, Ducharme, Vincent and Jauvin, A Neural Probabilistic Language Model (longer JMLR version)
Fri, Feb 21, 2020 Drop Deadline
Mon, Feb 24, 2020 Part-of-Speech Tagging [slides] [video] Jurafsky and Martin, Chapter 8 "Part-of-Speech Tagging"
Wed, Feb 26, 2020 Constituency Grammars [slides] [video] Jurafsky and Martin, Chapter 12 "Constituency Grammars"
Mon, Mar 2, 2020 Grammars and Constituency Parsing [slides] [video] [video-part-2] Jurafsky and Martin, Chapter 12 "Constituency Grammars"
Jurafsky and Martin, Chapter 13 "Constituency Parsing"
Lisbon Machine Learning School, CKY Demo
Dragomir Radev, (Video) Classic Parsing Methods for Natural Language Processing
Dragomir Radev, (Video) The Penn Treebank
Wed, Mar 4, 2020 Statistical Constituency Parsing [slides] [video] Jurafsky and Martin, Chapter 14 "Statistical Constituency Parsing"
Mon, Mar 9, 2020 No Lecture - Spring Break
Wed, Mar 11, 2020 No Lecture - Spring Break
Mon, Mar 16, 2020 No Lecture - Spring Break Extended
Wed, Mar 18, 2020 No Lecture - Spring Break Extended
Mon, Mar 23, 2020 Updates to course policies / Wrap-up of Constituency Parsing / Dependency Parsing [slides] [video] Jurafsky and Martin, Chapter 14 "Statistical Constituency Parsing"
Jurafsky and Martin, Chapter 15 "Dependency Parsing"
Dragomir Radev, Dependency Parsing
Dragomir Radev, Statistical Parsing
Wed, Mar 25, 2020 Logical Representations of Sentence Meaning [slides] [video] Jurafsky and Martin, Chapter 16 "Logical Representations of Sentence Meaning"
Wed, Mar 25, 2020 HW7 "Named Entity Recognition" due
Mon, Mar 30, 2020 Wrapup of Logical Representations of Sentence Meaning and Information Extraction [slides] [video] Jurafsky and Martin, Chapter 18 "Information Extraction"
Mon, Mar 30, 2020 Quiz due (covers Chapters 12-14) Jurafsky and Martin, Chapter 12 "Constituency Grammars"
Jurafsky and Martin, Chapter 13 "Constituency Parsing"
Jurafsky and Martin, Chapter 14 "Statistical Constituency Parsing"
Wed, Mar 25, 2020 Deadline to decide on term project versus weekly homework option. Please specify your preference by filling out this form.
Fri, Mar 27, 2020 Deadline to complete the IRB training for HW9, if you're doing the HW option. Please follow the instruction for obtaining data on the HW9 page.
Wed, Apr 1, 2020 Information Extraction and Semantic Role Labeling [slides] [video] Jurafsky and Martin, Chapter 20 "Semantic Role Labeling"
Wed, Apr 1, 2020 HW8 "Learning Hypernyms" due
Fri, Apr 3, 2020 Withdraw Deadline
Mon, Apr 6, 2020 Semantic Role Labeling and Encoder-Decoder Models [slides] [video] Jurafsky and Martin, Chapter 20 "Semantic Role Labeling"
Jurafsky and Martin, Chapter 10 "Encoder-Decoder Models, Attention, and Contextual Embeddings"
Mon, Apr 6, 2020 Quiz due (covers Chapters 15 and 16) Jurafsky and Martin, Chapter 15 "Dependency Parsing"
Jurafsky and Martin, Chapter 16 "Logical Representations of Sentence Meaning"
Wed, Apr 8, 2020 Encoder-Decoder Models and Neural Machine Translation [slides] [video] Graham Neubig, Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
Wed, Apr 8, 2020 Neural Machine Translation (recording of guest lecture from 2019) [slides] [video] by Graham Neubig Graham Neubig, Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
Wed, Apr 8, 2020 Project Milestone 1 or HW9 "Classify Depression" due
Mon, Apr 13, 2020 MT Evaluation and Cross-lingual Models of Word Embeddings [slides] [video] Shyam Upadhyay, Manaal Faruqui, Chris Dyer, Dan Roth, Cross-lingual Models of Word Embeddings: An Empirical Comparison
Mon, Apr 13, 2020 Quiz due (covers Chapters 18 and 20) Jurafsky and Martin, Chapter 18 "Information Extraction"
Jurafsky and Martin, Chapter 20 "Semantic Role Labeling"
Wed, Apr 15, 2020 Dialogue Systems and Chatbots [slides] [video] Jurafsky and Martin, Chapter 26 "Dialogue Systems and Chatbots"
Wed, Apr 8, 2020 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [slides] [video] by Jacob Devlin Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT&colon Pre-training of Deep Bidirectional Transformers for Language Understanding
Wed, Apr 15, 2020 Project Milestone 2 or HW10 "Neural Machine Translation" due
Mon, Apr 20, 2020 Text Generation with Neural Networks [slides] [video] by Daphne Ippolito Optional: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin, Attention is All You Need
Optional: Jay Alammar, The Illustrated Transformer
Optional: Jay Alammar, The Illustrated GPT-2 (Visualizing Transformer Language Models)
Optional: Jay Alammar, The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
Optional: Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng, The Woman Worked as a Babysitter: On Biases in Language Generation
Mon, Apr 20, 2020 Quiz due (covers Chapters 10 and 26) Jurafsky and Martin, Chapter 10 "Encoder-Decoder Models, Attention, and Contextual Embeddings"
Jurafsky and Martin, Chapter 26 "Dialog Systems and Chatbots"
Wed, Apr 22, 2020 The Transformer Model [slides] [video] by Daphne Ippolito Optional: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser and Illia Polosukhin, Attention is All You Need
Optional: Jay Alammar, The Illustrated Transformer
Optional: Jay Alammar, The Illustrated GPT-2 (Visualizing Transformer Language Models)
Optional: Jay Alammar, The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
Wed, Apr 22, 2020 Project Milestone 3 or HW11 "Perspective Detection" due
Mon, Apr 27, 2020 Question Answering [slides] [video] Jurafsky and Martin, Chapter 25 "Question Answering"
Mon, Apr 27, 2020 Quiz due (covers Chapters 22 and 25) Jurafsky and Martin, Chapter 22 "Coreference Resolution"
Jurafsky and Martin, Chapter 25 "Question Answering"
Wed, Apr 29, 2020 Course Wrapup [slides] [video]
Wed, Apr 29, 2020 Project Milestone 4 or HW12 "BERT" due
Wed, Apr 29, 2020 Deadline to switch to Pass/Fail