Date

Topic

Reading Due

Assignment

8/30  Introduction to class  
9/1  Syllabus, Introduction  The Discipline of Machine Learning (6 pagse)
Sections 1  1.3 of Sutton and Barto (7 pages)  
9/3  Decision Tree Learning  
9/6  More Decision Tree Learning
Least Mean Squares  
9/8  Perceptrons and Artifical Neural Networks  
9/10  Artifical Neural Networks  
9/13  Bayes Theorem  
9/15  Naive Bayes  
9/17  Bayes Nets  
9/20  Bayes Nets + Intro to COLT  
9/22  Version Spaces and PAC  Reading Response on Naive Bayes due (on Moodle)  
9/24  (In)finite Hypothesis Spaces  Reading Response on Neural Nets due (on Moodle)  
9/27  Sutton + Barto, Chapter 2: Gambling!  
9/29  S&B, Chapter 3: The RL framework  
10/1  Bellman Equations  
10/4  Policy Evaluation, Policy Iteration  
10/6  Value Iteration  Reading Response: S&B sections 2.1, 2.2, 2.4, 2.5, 2.7, 2.8, 2.11. Due by 6am.  
10/8  S&B, Chapter 5: Monte Carlo  
10/11  No class  Fall Break
 
10/13  More Monte Carlo: Policy Evaluations  
10/15  Offline MC and TD  S&B Chapter 5. Due 6am Monday 10/18.  
10/18  Sarsa  
10/20  Qlearning  
10/22  Discussion of project 0, project 1, and Keepaway  Project 1, Step 0. Install rlcompetition code. Hack the getAction() function in agents/marioAgentJava/src/edu/rutgers/rl3/comp/ExMarioAgent.java to always go right and send me the code of this function by 6am Friday.  
10/25  Eligibility Traces  S&B Chapter 6. Due 6am Wednesday 10/18. Instead of a summary, you can answer the following two questions: "How would you explain the difference between Dynamic Programming, Monte Carlo, and Temporal Difference Learning? When would you use one of these methods instead of another?"  Project 1, checkpoint #1 due 
10/27  Eligibility Traces  on the board  
10/29  More Eligibility  Response Due Monday, 6am: Read 7.0, 7.1, 7.2, 7.3, 7.5, 7.8, and 7.9  
11/1  Function Approximation  Project 1, checkpoint #2 due  
11/3  Function Approximation 2, The Remix  Read your section for Friday  
11/5  Finishing off Function Approximation and Discussion of paper  
11/8  Planning and Learning  
11/10  Planning and Learning, rest of chapter  Skim S&B chapter 8, read section 8.4 (no response due  project on Monday!)  
11/13  RMax and RLDT  
11/15  Shaping and TAMER  
11/17  More Shaping, X2  6am, Monday the 22nd: Send Matt a proposal for Project 3. Suggestions: One of the topics mentioned in class (see 11/17 slides), using some of the UCI data and decision trees, or one of the RL topics in Mario, Tetris, or Mountain Car. Or anything you think of! Also, send Matt suggestions for what you'd find most interesting to discuss in the final 2 weeks of class.  
11/19  Guest Lecture: GAs  
11/22  Ensemble Methods  
11/24 
No class  Thanksgiving Break
 
11/26 
No class  Thanksgiving Break
 
11/29  InstanceBased Methods  
12/1  No lecture  Matt was sick  
12/3  Hierarchical RL  
12/6  Transfer Learning  
12/8  Discussion of Intrinsic Rewards  
12/10  Discussion of Helicopter Flight  